System with endoscope and image sensor and method for processing medical images

ABSTRACT

A system includes an endoscope including a scope and an image sensor. The image sensor is configured to capture medical image data that includes effective image portion data and a mechanical vignetting portion data, the mechanical vignetting portion data of the medical image data being generated due to mechanical vignetting caused by a difference in the image sensor which generates the medical image data and the scope. There is also circuitry configured to determine evaluation information from image data which is from the effective image portion data, and execute a control process to at least partially control at least one of an autofocus processing, and an auto white balance processing on the endoscope on the basis of the evaluation information.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is based on PCT filing PCT/JP2019/009853, filedMar. 12, 2019, which claims priority to JP 2018-051955, filed Mar. 20,2018, the entire contents of each are incorporated herein by reference.

TECHNICAL FIELD

The present technology relates to an endoscope system, a control method,an information processing device, and a program, and more particularly,to an endoscope system, a control method, an information processingdevice, and a program capable of photographing an endoscopic imagesuitable for a surgical operation.

BACKGROUND ART

In recent years, an endoscopic surgical operation having lowinvasiveness has attracted attention. In order to obtain an endoscopicimage suitable for the surgical operation, an endoscope having anautofocus (AF) function, an automatic exposure (AE) function, and anauto white balance (AWB) function has been proposed.

For example, PTL 1 discloses a technique of performing the AF functionof the endoscope by calculating a focus evaluation value on the basis ofthe endoscopic image.

CITATION LIST Patent Literature

PTL 1: JP 2013-80108A

SUMMARY OF INVENTION Technical Problem

Typically, in the vicinity of a periphery of the endoscopic image, amask region as a significantly dark area is generated due to an opticalshadow of the scope (vignetting). For example, in a case where the AF isperformed on the basis of such an endoscopic image, focusing may beperformed on the mask region in some cases.

In addition, a position or size of the mask region changes depending onthe type of the employed scope. Therefore, it is difficult to fix acalculation target region for the evaluation value in advance.

In view of the aforementioned circumstances, the present technology hasbeen made to photograph an endoscopic image suitable for a surgicaloperation.

Solution to Problem

A system includes an endoscope including a scope and an image sensor.The image sensor is configured to capture medical image data thatincludes effective image portion data and a mechanical vignettingportion data, the mechanical vignetting portion data of the medicalimage data being generated due to mechanical vignetting caused by adifference in the image sensor which generates the medical image dataand the scope. There is also circuitry configured to determineevaluation information from image data which is from the effective imageportion data, and execute a control process to at least partiallycontrol at least one of an autofocus processing, and an auto whitebalance processing on the endoscope on the basis of the evaluationinformation.

Further, there is a method of processing medical image information whichincludes determining evaluation information using effective imageportion data of medical image data, the medical image data including theeffective image portion data and mechanical vignetting portion data, themechanical vignetting portion data of the medical image data beinggenerated due to mechanical vignetting caused by a difference in animage sensor which generates the medical image data and a medicalinstrument. Additionally, there is an executing of a control processincluding at least one of an autofocus process, or an auto white balanceprocess on the basis of the evaluation information.

In the present technology, an effective region of the scope is detectedfrom a photographic image photographed by the image sensing device, anda control process including at least one of an autofocus processing, anautomatic exposure processing, or an auto white balance processing isexecuted on the basis of the evaluation value of the effective region.

Advantageous Effects of Invention

In the present technology, it is possible to photograph an endoscopicimage suitable for a surgical operation. Moreover, the presenttechnology permits a system which addresses issues with vignetting andprovides advantageous focusing and/or white balance processing.

Note that the advantageous effects described herein are not necessarilylimited, but may include any one of those described in this disclosure.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram illustrating an exemplary configuration of anendoscope system according to an embodiment of the present technology.

FIG. 2 is a perspective view illustrating appearance of an endoscope.

FIG. 3 is a diagram illustrating an exemplary photographic image.

FIG. 4 is a diagram illustrating exemplary photographic images.

FIG. 5 is a block diagram illustrating exemplary configurations of a CCUand an endoscope.

FIG. 6 is a diagram illustrating an exemplary setting of samplingframes.

FIG. 7 is a diagram illustrating exemplary edge detection of a maskregion.

FIG. 8 is a diagram illustrating exemplary edge detection of a maskregion.

FIG. 9 is a diagram illustrating an exemplary setting of sampling framesin a case where the mask region is detected on the basis of the AFevaluation value.

FIG. 10 is a diagram illustrating another exemplary setting of samplingframes in a case where a mask region is detected on the basis of the AFevaluation value.

FIG. 11 is a diagram illustrating further another exemplary setting ofsampling frames in a case where a mask region is detected on the basisof the AF evaluation value.

FIG. 12 is a diagram illustrating an exemplary setting of an evaluationvalue calculation target region.

FIG. 13 is a diagram illustrating another exemplary setting of theevaluation value calculation target region.

FIG. 14 is a flowchart for describing an AF processing of the CCU.

FIG. 15 is a flowchart for describing a mask region detection processingperformed in step S3 of FIG. 14.

FIG. 16 is a flowchart for describing another mask region detectionprocessing performed in step S3 of FIG. 14.

FIG. 17 is a block diagram illustrating an exemplary configuration ofthe CCU for an AE processing.

FIG. 18 is a diagram illustrating an exemplary setting of the evaluationvalue calculation target region for the AE evaluation value.

FIG. 19 is a diagram illustrating another exemplary setting of theevaluation value calculation target region for the AE evaluation value.

FIG. 20 is a block diagram illustrating an exemplary configuration of acomputer.

DESCRIPTION OF EMBODIMENTS

Embodiments of the present technology will now be described. Thedescription will be made in the following sequence.

1. Configuration of Endoscope System

2. First Embodiment: Example of Detecting Mask Region on the basis ofSampling Value

3. Second Embodiment: Example of Detecting Mask Region on the basis ofAF Evaluation Value

4. Third Embodiment: First Exemplary Setting of Evaluation ValueCalculation Target Region

5. Fourth Embodiment: Second Exemplary Setting of Evaluation ValueCalculation Target Region

6. Fifth Embodiment: Exemplary Autofocus processing

7. Operation of CCU

8. Other Examples

<<Configuration of Endoscope System>>

FIG. 1 is a diagram illustrating an exemplary configuration of anendoscope system according to an embodiment of the present technology.

The endoscope system 1 of FIG. 1 includes a display device 11, a cameracontrol unit (CCU) 12, a light source device 13, a treatment tool device14, a pneumoperitoneum device 15, a recorder 16, and a printer 17mounted on a cart 18.

In addition, the endoscope system 1 includes an endoscope 19, an energytreatment tool 20, a forceps 21, trocars 22 to 25, a foot switch 26, anda patient's bed 27. The endoscope system 1 is installed, for example, ina surgical operation room to assist an operator who performs alaparoscopic surgery for a lesion inside an abdomen 31 of a patientlying on the patient's bed 27.

The display device 11 includes a stationary two-dimensional display, ahead-mount display, or the like. The display device 11 displays an imageof a surgical treatment portion (surgical field area) supplied from theCCU 12 and the like.

The CCU 12 is connected to each part such as the light source device 13and the endoscope 19. The CCU 12 receives the photographic image of thesurgical treatment portion photographed by the endoscope 19 andtransmitted through a camera cable and displays it on the display device11. The CCU 12 outputs the photographic image photographed by theendoscope 19 to the recorder 16 or the printer 17 as necessary. Notethat the CCU 12 and the endoscope 19 may be connected to each other viawireless communication.

In addition, the CCU 12 performs an autofocus processing as a processingfor performing autofocus (AF) of the endoscope 19. That is, in theendoscope system 1, focus adjustment of the endoscope 19 isautomatically performed under control of the CCU 12 regardless of anoperator's manipulation.

The light source device 13 is connected to the endoscope 19 through alight guide cable. The light source device 13 outputs light beams havingvarious wavelengths to the endoscope 19 by switching them.

The treatment tool device 14 as a high-frequency output device isconnected to the energy treatment tool 20 and the foot switch 26 througha cable. The treatment tool device 14 outputs a high-frequency currentto the energy treatment tool 20 in response to a manipulation signalsupplied from the foot switch 26.

The pneumoperitoneum device 15 has an air blower unit and an air suctionunit. The pneumoperitoneum device 15 blows the air from a hole of thetrocar 24 as an opening tool installed in an abdominal wall of anabdomen 31 to the inside of the abdomen 31.

The recorder 16 records a photographic image supplied from the CCU 12.

The printer 17 prints the photographic image supplied from the CCU.

The endoscope 19 is inserted into the abdomen 31 from the hole of thetrocar 22 installed in the abdominal wall of the abdomen 31. Theendoscope 19 irradiates the inside of the abdomen 31 with the lightemitted from the light source device 13 to photograph the inside of theabdomen 31. The endoscope 19 outputs the photographic image obtained byphotographing the inside of the abdomen 31 to the CCU 12.

The energy treatment tool 20 includes an electrocautery or the like. Theenergy treatment tool 20 is inserted into the abdomen 31 from the holeportion of the trocar 23 installed in the abdominal wall of the abdomen31. The energy treatment tool 20 modifies or removes an inner part ofthe abdomen 31 using electric heat.

The forceps 21 is inserted into the abdomen 31 from the hole portion ofthe trocar 25 installed in the abdominal wall of the abdomen 31. Theforceps 21 is used to grip an inner part of the abdomen 31. Theendoscope 19, the energy treatment tool 20, and the forceps 21 aregripped by an operator, assistant, scopist, robot, or the like.

The foot switch 26 receives a foot manipulation from an operator,assistant, or the like. The foot switch 26 outputs a manipulation signalindicating the received manipulation to the CCU 12 or the treatment tooldevice 14.

The operator may excise a lesion inside the abdomen 31 while looking atthe photographic image displayed on the display device 11 by using theendoscope system 1.

FIG. 2 is a perspective view illustrating appearance of the endoscope19.

As illustrated in FIG. 2, the endoscope 19 as a rigid endoscope includesa camera head 51 and a scope 52 having a long lens tube. The endoscopesare classified into a flexible endoscope whose body insertion part isbendable and a rigid endoscope whose body insertion part is unbendable.The endoscope 19 is the latter endoscope. The operator performs asurgical operation by gripping the camera head 51 and inserting thescope 52 into the patient's body.

The camera head 51 internally has an imaging element that performsphotoelectric conversion of the light guided by the lens of the scope 52from the living body, a driving unit for driving the lens of the scope52, and the like. The camera head 51 irradiates the inside of theabdomen 31 by guiding the light emitted from the light source device 13via the scope 52 to photograph the surgical treatment portion. Thecamera head 51 outputs the photographic image obtained from thephotographing to the CCU 12 through the camera cable.

The scope 52 is detachably installed in the camera head 51. The scope 52refers to a plurality of types of scopes having different specificationssuch as scopes having different diameters (scope diameters) or scopeshaving different F-numbers of lenses. Which type of scope 52 is employedis appropriately selected depending on details of the surgicaloperation, a condition of the surgical treatment portion, and the like,and the selected scope 52 is installed in the camera head 51.

FIG. 3 is a diagram illustrating an exemplary photographic image.

As illustrated in FIG. 3, a circular effective region is formedapproximately in the center of an oblong rectangular photographic image,and a dark mask region is formed outward of the effective region.

The mask region is a region formed by an optical shadow (vignetting) ofthe scope 52. More specifically, vignetting or mechanical vignetting iscaused by a difference in an image sensor such as the imaging element111 which generates medical image data and the scope 52. Since the scope52 has a long cylindrical shape, a dark region is imaged around theimage circle. The condition of the surgical treatment portion isdisplayed within a range of the effective region having no vignetting.Note that vignetting caused by the scope 52 refers to an optical shadow,for example, generated as an optical path is physically blocked by aside wall of the scope 52.

As described above, the scope 52 is switchable depending on details ofthe surgical operation or the like. A position of the effective region,a size of the effective region, a shape of the effective region, and thelike on the photographic image are changed depending on the type of thescope 52 installed in the camera head 51.

FIGS. 4A and 4B are diagrams illustrating exemplary photographic images.In FIGS. 4A and 4B, the condition of the surgical treatment portioninside the effective region is omitted intentionally for simplicitypurposes. Effective image portion data is image data that is obtainedfrom, around, or near the effective region. Mechanical vignettingportion data is image data that is obtained from, around, or near themask region which is a region formed by an optical shadow (vignetting)of the scope 52.

The photographic image of FIG. 4A is a photographic image obtained in acase where the scope 52 having a scope diameter (diameter) shorter thana vertical length of the photographic image is installed. Similar to thephotographic image of FIG. 3, the photographic image of FIG. 4A includesthe circular effective region as a whole.

Meanwhile, the photographic image of FIG. 4B is a photographic imageobtained in a case where the scope 52 having a scope diameter longerthan the vertical length of the photographic image is installed. Theeffective region of FIG. 4B has a circular shape truncated in the upperend lower ends.

In this manner, an effective region having a diameter corresponding tothe scope diameter is formed on the photographic image. A size of thediameter of the scope 52 installed in the camera head 51 can bespecified on the basis of the size of the diameter of the effectiveregion included in the photographic image.

As described above, focus adjustment of the endoscope 19 of theendoscope system 1 is automatically performed using the AF.

Assuming that the AF evaluation value as an evaluation value forperforming the AF is calculated for the entire photographic image, theAF evaluation values of each region including the mask region arecalculated. This is not desirable. Therefore, in a case where the AFevaluation values are calculated for each region including the maskregion, for example, an operation for focusing on the edge (border) ofthe mask region is performed. Evaluation information is one or moreevaluation values or other types of information. Evaluation informationincludes at least information which comes from effective image portiondata.

In addition, assuming that the calculation target region for the AFevaluation value is fixedly set inside the effective region, the targetregion is restricted to a narrow range of the vicinity of the center ofthe photographic image corresponding to the effective region regardlessof which scope 52 is installed. As illustrated in FIGS. 4A and 4B, therange of the effective region changes depending on the type of theinstalled scope 52.

In the CCU 12, the diameter of the scope 52 installed in the endoscope19, the center position of the effective region, and the like arespecified on the basis of the photographic image, and the evaluationvalue calculation target region as a region for calculating the AFevaluation value is set inside the effective region on the basis of thespecified information. In addition, the AF evaluation value iscalculated for the evaluation value calculation target region to performthe autofocus processing.

Since the AF evaluation value is calculated for the evaluation valuecalculation target region set within the effective region to perform theautofocus processing, the CCU 12 can reliably focus on the surgicaltreatment portion imaged on the effective region.

A series of processes of the CCU 12 for performing the autofocusprocessing by setting the evaluation value calculation target regiondepending on the scope 52 installed in the endoscope 19 in this mannerwill be described below.

First Embodiment: Example of Detecting Mask Region on the Basis ofSampling Value

<Exemplary Configurations of CCU and Endoscope>

FIG. 5 is a block diagram illustrating exemplary configurations of theCCU 12 and the endoscope 19.

As illustrated in the left half of FIG. 5, the camera head 51 includesan imaging element 111, an imaging element driver 112, a lens driver113, a zoom lens driving unit 114, and a focus lens driving unit 115.

The imaging element 111 includes, for example, a CMOS image sensor or aCCD image sensor. The imaging element 111 converts an optical imagefocused on an imaging surface into an electric signal by photoelectricconversion and outputs the electric signal as a photographic signal tothe CCU 12.

The imaging element driver 112 is a driver for driving the imagingelement 111. The imaging element driver 112 allows the imaging element111 to perform a predetermined operation such as a photographingoperation or a reset operation by outputting a drive signal. Forexample, a shutter speed of the imaging element 111 is controlled, forexample, by the drive signal output from the imaging element driver 112.

The lens driver 113 includes a processor such as a central processingunit (CPU) or a digital signal processor (DSP). The lens driver 113controls the operations of the zoom lens driving unit 114 and the focuslens driving unit 115 depending on a control signal supplied from theCCU 12.

The zoom lens driving unit 114 adjusts a photographic magnificationratio by moving the zoom lens 101 of the scope 52 along an optical axis.

The focus lens driving unit 115 performs focus adjustment by moving thefocus lens 102 of the scope 52 along an optical axis.

As illustrated in the right half of FIG. 5, the CCU 12 includes a camerasignal processing unit 131, a sampling frame gate 132, a sampling unit133, a mask detection unit 134, an AF sampling unit 135, and a lenscontroller 136.

The camera signal processing unit 131 applies various types of signalprocessings such as a white balance processing and a γ correctionprocessing to the photographic signal supplied from the imaging element111.

The camera signal processing unit 131 outputs the photographic signalobtained through the signal processing to the display device 11 as avideo signal. The image of the surgical treatment portion is displayedon the display device 11 on the basis of the video signal output fromthe camera signal processing unit 131. The photographic signal outputfrom the camera signal processing unit 131 is also supplied to thesampling frame gate 132.

The sampling frame gate 132 sets a sampling frame in a predeterminedregion on the photographic image under control of the lens controller136. For example, the sampling frame is set in a predetermined region ofthe photographic image including the mask region before detection of themask region. In addition, the sampling frame is set in the effectiveregion after detection of the mask region.

The sampling frame gate 132 outputs a photographic signal of the pixelsof the sampling frames out of the photographic signals supplied from thecamera signal processing unit 131. The photographic signal output fromthe sampling frame gate 132 is supplied to the sampling unit 133 and theAF sampling unit 135.

The sampling unit 133 performs sampling for the photographic signalsupplied from the sampling frame gate 132 and outputs sampling values ofeach sampling frame to the mask detection unit 134. For example, thesampling unit 133 integrates luminance values of pixels of each samplingframe to obtain a result of the integration as the sampling value.

The mask detection unit 134 detects an edge of the mask region on thebasis of the sampling value supplied from the sampling unit 133.Although the mask detection unit 134 detects the edge of the mask regionin this description, the edge detection of the mask region also includesedge detection of the effective region.

The mask detection unit 134 specifies a diameter of the scope 52installed in the camera head 51 on the basis of a position of thedetected edge of the mask region. In addition, the mask detection unit134 specifies a center position of the effective region and a position(range) of the mask region on the basis of the edge position of the maskregion.

The mask detection unit 134 outputs information regarding the scopediameter, the center position of the effective region, and the positionof the mask region to the lens controller 136 as a result of the maskdetection. As described below, detection of the mask region using themask detection unit 134 may also be performed on the basis of the AFevaluation value obtained by the AF sampling unit 135 in some cases.

The AF sampling unit 135 calculates the AF evaluation value on the basisof the photographic signal supplied from the sampling frame gate 132.For example, the AF sampling unit 135 calculates the AF evaluation valuerepresenting contrast by calculating a second-order derivative for theluminance signals of all pixels of the AF sampling frames. Note that, ingeneral, a difference of the luminance signal between neighboring pixelsincreases, and the contrast increases in a focused state, compared to anunfocused state.

The AF evaluation value calculated by the AF sampling unit 135 issupplied to the lens controller 136 and is also supplied to the maskdetection unit 134 appropriately.

The lens controller 136 adjusts a position of the zoom lens 101 byoutputting the control signal to the lens driver 113 of the camera head51.

The lens controller 136 also outputs the control signal to the lensdriver 113 of the camera head 51 and adjusts a position of the focuslens 102. A process of adjusting the position of the focus lens 102 byoutputting the control signal corresponds to the autofocus processingfor performing the AF of the endoscope 19. The control signal in thiscase is a signal containing at least a position of the focus lens 102 ofthe lens driver 113 or a displacement of a position of the focus lens102 as an AF control parameter.

The lens controller 136 adjusts a shutter speed or an ISO sensitivity ofthe imaging element 111 by appropriately outputting a control signal tothe imaging element driver 112 of the camera head 51. A process ofadjusting the shutter speed or the ISO sensitivity by outputting thecontrol signal and controlling exposure regardless of an operator'smanipulation corresponds to the automatic exposure processing forperforming the AE. As described below, the AE function is implemented onthe basis of the detection result of the mask region. The control signalin this case is a signal containing at least the speed value of theshutter speed or the sensitivity value of the ISO sensitivity as an AEcontrol parameter.

A focus command signal, a zoom command signal, a manual/automatic focusconversion signal, and the like are input to the lens controller 136 inresponse to an operator's manipulation. The zoom command signal is asignal representing details of the zoom adjustment performed by theoperator, and the focus command signal is a signal representing detailsof the focus adjustment performed by the operator.

The manual/automatic focus conversion signal is a conversion signal forindicating whether the focus adjustment is performed in a manual mode orin an autofocus mode. The focus command signal is input in a case wherean option for performing the focus adjustment in the manual mode isselected. The CCU 12 and/or the camera head 51 are illustrated anddescribed as including various units, processors, CPUs, controllers,drivers, etc. These elements and other elements of the invention can beimplemented using circuitry configured to perform the functionalitydisclosed herein. Moreover, in some embodiments, electronic circuitry orprocessing circuitry including, for example, a microprocessor,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute computer readable programinstructions by using information from the computer readable programinstructions to configure or customize the electronic circuitry, inorder to perform aspects of the present disclosure.

<Detection of Mask Region>

FIG. 6 is a diagram illustrating an exemplary setting of the samplingframe.

The sampling frames are set, for example, across the entire area of thephotographic image. In the example of FIG. 6, thirty sampling frames setin a column (vertical) direction and forty sampling frames set in a row(horizontal) direction are arranged in a matrix shape across the entirearea of the photographic image. A total size of the sampling frames inFIG. 6 is equal to the size of the photographic image.

The mask detection unit 134 detects an edge of the mask region on thebasis of the a thousand and two hundred (1200) sampling values obtainedfrom the photographic signal of the pixels of each sampling frame.

In this manner, the sampling frames are set for the photographic imagesuch that a resolution sufficient to detect an edge of the mask regioncan be provided.

Note that, although the total size of the sampling frames is equally setto the size of photographic image in the example of FIG. 6, the size ofthe sampling frames may be set to a range smaller than the size of thephotographic image. In addition, the number of the sampling frames mayalso be set arbitrarily.

FIG. 7 is a diagram illustrating an example of the edge detection in themask region.

A series of letters “x” inserted into each sampling frame respectivelyrefer to predetermined sampling values. Sampling frames having anumerical value “0” refer to sampling frames having a sampling value ofzero, which means a sampling frame including a black pixel.

In a case where the sampling values are obtained by the sampling unit133 in this manner, the mask detection unit 134 of FIG. 5 obtains amaximum sampling value out of the sampling values of the sampling framesof each column and a maximum sampling value out of the sampling valuesof the sampling frames of each row.

In addition, the mask detection unit 134 obtains differences betweensampling values of every other column in a sequence of maximum samplingvalues of each column as illustrated before a void arrow #1. Similarly,for the row direction, the mask detection unit 134 obtains differencesbetween sampling values of every other row in a sequence of maximumsampling values of each row.

For example, the mask detection unit 134 sequentially calls thedifferences between the sampling values of every other column startingfrom the leftmost difference and detects a position of the column wherea difference equal to or larger than a threshold value is obtained as aleft edge position of the mask region. In addition, the mask detectionunit 134 sequentially calls the differences between the sampling valuesof every other column starting from the rightmost difference and detectsa position of the column where a difference equal to or larger than athreshold value is obtained as a right edge position of the mask region.

Similarly, for the row direction, the mask detection unit 134sequentially calls the differences between the sampling values of everyother row starting from the uppermost difference and detects a positionof the row where a difference equal to or larger than threshold value isobtained as an upper edge position of the mask region. In addition, themask detection unit 134 sequentially calls the differences between thesampling values of every other row starting from the lowermostdifference and detects a position of the row where a difference equal toor larger than a threshold value is obtained as a lower edge position ofthe mask region.

On the basis of the upper, lower, left, and right edge positions and afact that the effective region has a circular shape, the mask detectionunit 134 detects all edges surrounding the effective region asillustrated in FIG. 8.

In FIG. 8, the hatched sampling frames are sampling frames of thepositions detected as edges of the mask region. The sampling framespositioned outward of the hatched sampling frames belong to the maskregion, and the inward sampling frames (including the hatched samplingframes) belong to the effective region.

Note that, in a case where the effective region has a large diameter,and the upper and lower ends of the effective region are truncated, theleft and right edges are detected on the basis of the differences of thesampling values of every other column to specify the mask region. Inaddition, in a case where no edge is detected, it is specified thatthere is no mask region.

In a case where the mask region and the effective region are detected,the mask detection unit 134 specifies, for example, an average edgewidth as a diameter of the effective region, that is, as a scopediameter on the basis of the edge width of the effective region. Inaddition, the mask detection unit 134 specifies a position indicated byan average median value of the edge width as a center position of theeffective region. Note that a table where a range of the average edgewidth and corresponding scope diameters are associated with each othermay be stored in advance, and the scope diameter may be specified byreferencing the table on the basis of the obtained average edge width.

Information indicating the scope diameter and the center position of theeffective region is supplied to the lens controller 136 along withinformation regarding the position of the mask region. The lenscontroller 136 performs a setting of sampling frames used in calculationof the AF evaluation value and the like on the basis of the detectionresult of the mask region.

Second Embodiment: Example of Detecting Mask Region on the Basis of AFEvaluation Value

The mask region may be detected on the basis of the AF evaluation valueobtained by the AF sampling unit 135 instead of the sampling valueobtained by the sampling unit 133.

First Exemplary Setting

FIG. 9 is a diagram illustrating an exemplary setting of the samplingframes in a case where the mask region is detected on the basis of theAF evaluation value.

As indicated by the sampling frames F1 to F4 of the left photographicimage of FIG. 9, the sampling frame gate 132 sets sampling frames, forexample, having a narrow strip shape in the upper, lower, left, andright ends of the photographic image. The sampling frames F1 to F4 areset, for example, under control of the lens controller 136.

The sampling frames F1 and F2 indicated by one-dotted chain lines aresampling frames for detecting upper and lower edges, respectively, ofthe mask region. In addition, the sampling frames F3 and F4 indicated bydotted lines are sampling frames for detecting left and right edges,respectively, of the mask region.

In addition, as indicated by the void arrows, the sampling frame gate132 moves positions of the sampling frames F1 to F4 toward a center ofthe photographic image.

The AF sampling unit 135 sequentially calculates the AF evaluationvalues on the basis of the photographic signals corresponding to thepixels within the sampling frames set by changing the positions in thismanner.

In a case where a change equal to or larger than a threshold value isgenerated in the AF evaluation value calculated by the AF sampling unit135, the mask detection unit 134 specifies the position where the changeis detected as an edge position of the mask region. For example, asindicated before the arrow #2, when each of the sampling frames F1 to F4is set as the edge position of the mask region, a change of the AFevaluation value equal to or larger than the threshold value isdetected.

The mask detection unit 134 specifies a diameter of the scope installedin the camera head 51, a center position of the effective region, and aposition of the mask region on the basis of a distance L1 between thesampling frames F1 and F2 and a distance L2 between the sampling framesF3 and F4. For example, an average value of the distances L1 and L2 isspecified as the scope diameter, and a position indicated by an averagemedian value of the edge is specified as a center position of theeffective region.

Second Exemplary Setting

FIG. 10 is a diagram illustrating another exemplary setting of thesampling frame in a case where the mask region is detected on the basisof the AF evaluation value.

The method of setting the sampling frame of FIG. 10 is different fromthe method of setting the sampling frame of FIG. 9 in that the samplingframes F1 to F4 are set such that the width is sequentially widenedinstead of changing the positions of the sampling frames F1 to F4. Thewidths of the sampling frames F1 to F4 are sequentially widened bymoving inner edges of the narrow strip-like sampling frames F1 to F4 setin the upper, lower, left, and right ends of the photographic imagetoward a center of the photographic image as indicated by the voidarrows.

The AF sampling unit 135 sequentially calculates the AF evaluation valueon the basis of the photographic signals corresponding to the pixels ofthe sampling frames set by changing the width.

In a case where a change equal to or larger than a threshold valueoccurs in the AF evaluation value calculated by the AF sampling unit135, the mask detection unit 134 specifies an inner edge position of thesampling frame where the change is detected as the edge position of themask region. For example, as indicated before the arrow #3, a change ofthe AF evaluation value equal to or larger than the threshold value isdetected when the inner edges of the sampling frames F1 to F4 are set asthe respective edge positions of the mask region.

The mask detection unit 134 specifies the diameter of the scopeinstalled in the camera head 51, the center position of the effectiveregion, and the position of the mask region on the basis of the distanceL1 between the sampling frames F1 and F2 and the distance L2 between thesampling frames F3 and F4.

Third Exemplary Setting

FIG. 11 is a diagram illustrating further another exemplary setting ofthe sampling frame in a case where the mask region is detected on thebasis of the AF evaluation value.

The method of setting the sampling frame of FIG. 11 is different fromthe setting methods of FIG. 9 and the like in that a plurality of smallsampling frames having an approximately square shape in an initial stateare set instead of the narrow strip-like sampling frames.

That is, as illustrated in the left half of FIG. 11, the sampling framegate 132 sets sampling frames F1-1 to F1-3 having an approximatelysquare shape in the upper end of the photographic image and setssampling frames F2-1 to F2-3 having an approximately square shape in thelower end of the photographic image. The sampling frame gate 132 setssampling frames F3-1 to F3-3 having an approximately square shape in theleft end of the photographic image and sets sampling frames F4-1 to F4-3having an approximately square shape in the right end of thephotographic image.

Furthermore, the sampling frame gate 132 moves positions of eachsampling frame toward the center of the photographic image or widens thewidth of the sampling frame toward the center of the photographic image.

The AF sampling unit 135 sequentially calculates the AF evaluationvalues on the basis of the photographic signals corresponding to thepixels of the sampling frames set by changing the position or the width.

In a case where a change equal to or larger than a threshold valueoccurs in the AF evaluation value calculated by the AF sampling unit135, the mask detection unit 134 specifies the position where the changeis detected as the edge position of the mask region.

For example, as illustrated before the arrow #4, a change of the AFevaluation value equal to or larger than the threshold value is detectedwhen the sampling frames F1-1 to F1-3 and F2-1 to F2-3 having anapproximately square shape are set as the respective edge positions ofthe mask region.

In addition, a change of the AF evaluation value equal to or larger thanthe threshold value is detected when the inner edges of the samplingframes F3-1 to F3-3 and F4-1 to F4-3 reach the respective edge positionsof the mask region.

The mask detection unit 134 specifies the diameter of the scopeinstalled in the camera head 51, the center position of the effectiveregion, and the position of the mask region on the basis of the distancebetween the opposite sampling frames.

Although, in the example of FIG. 11, the sampling frames F1-1 to F1-3and F2-1 to F2-3 set in the upper and lower ends of the photographicimage are set by changing the positions rather than the shapes, they mayalso be set by changing the widths. Furthermore, although the samplingframes F3-1 to F3-3 and F4-1 to F4-3 set in the left and right ends,respectively, of the photographic image are set by changing the widths,they may also be set by changing the positions rather than the shapes.

The method of setting the sampling frames by changing the shapes and themethod of setting the sampling frames by changing the widths may also beused in combination. It is possible to detect a specific shape of theedge of the mask region by changing the position or width of the smallsampling frame.

Note that, in a case where the diameter of the effective region islarge, and the effective region is vertically disconnected, the maskregion is detected only on the basis of the AF evaluation valuescalculated from the sampling frames set in the left and right sides. Inaddition, in a case where no edge is found, it is determined that thereis no mask region.

Information indicating the scope diameter specified from the AFevaluation value and the center position of the effective region issupplied to the lens controller 136 along with the information regardingthe position of the mask region. The lens controller 136 sets thesampling frame used in calculation of the AF evaluation value orperforms other controls on the basis of the detection result of the maskregion.

Third Embodiment: First Exemplary Setting of Evaluation ValueCalculation Target Region

FIGS. 12A and 12B are diagrams illustrating an exemplary setting of theevaluation value calculation target region.

As illustrated in FIGS. 12A and 12B, the lens controller 136 sets theevaluation value calculation target region serving as a target regionfor calculating the AF evaluation value inside the effective region soas not to overlap with the mask region. For example, the evaluationvalue calculation target region is set by magnifying or reducing a sizeof a default area and shifting a default center position on the basis ofthe detection result of the mask region.

The oblong area A1 of FIG. 12A is an evaluation value calculation targetregion set inside the effective region having a diameter shorter thanthe vertical length of the photographic image. The area A2 of FIG. 12Bis an evaluation value calculation target region set inside theeffective region having a diameter longer than the vertical length ofthe photographic image.

Any evaluation value calculation target region is set such that theentire area is included in the effective region. In FIGS. 12A and 12B,two oblong rectangles representing the evaluation value calculationtarget regions are inserted into the effective region in order to showthat any size can be set for the evaluation value calculation targetregion as long as it is included in the effective region.

Note that, although the evaluation value calculation target region hasan oblong rectangular shape in the example of FIGS. 12A and 12B, othershapes such as a square shape and a circular shape may also be employed.

Information regarding such an evaluation value calculation target regionis supplied from the lens controller 136 to the sampling frame gate 132,so that the sampling frame serving as a calculation target of the AFevaluation value is set inside the evaluation value calculation targetregion.

The AF sampling unit 135 calculates the AF evaluation value on the basisof the photographic signal of the pixel of the sampling frame set in theevaluation value calculation target region. In addition, the lenscontroller 136 also performs the autofocus processing on the basis ofthe calculated AF evaluation value.

As a result, the CCU 12 can reliably focus on the surgical treatmentportion imaged on the effective region.

Fourth Embodiment: Second Exemplary Setting of Evaluation ValueCalculation Target Region

FIGS. 13A and 13B are diagrams illustrating another exemplary setting ofthe evaluation value calculation target region.

Each segment of the mesh pattern overlapping on the photographic imageof FIGS. 13A and 13B represents a sampling frame. In the example ofFIGS. 13A and 13B, a plurality of sampling frames (eight sampling framesin the vertical direction and eight frames in the horizontal direction)are set side by side in a matrix shape around the center of thephotographic image. In this example, the positions of the samplingframes are fixed regardless of the range of the effective region. Thephotographic signals of the pixels of each sampling frame are suppliedfrom the sampling frame gate 132 to the AF sampling unit 135.

Out of the sampling frames fixedly set in this manner, the lenscontroller 136 selects the sampling frames entirely included in theeffective region without overlapping with the mask region as theevaluation value calculation target region serving as a calculationtarget for the AF evaluation value. The sampling frames colored in FIGS.13A and 13B are sampling frames selected as the evaluation valuecalculation target region. Meanwhile, the sampling frames overlappingwith the mask region are treated as invalid sampling frames (having aweight of zero).

The lens controller 136 outputs information regarding the sampling frameselected as the evaluation value calculation target region to the AFsampling unit 135 to calculate the AF evaluation value for the samplingframe selected as the evaluation value calculation target region.

That is, in the example of FIGS. 12A and 12B, the sampling framesserving as the calculation target of the AF evaluation value are set onthe basis of the detection result of the mask region. In comparison, inthe example of FIGS. 13A and 13B, the sampling frames serving as thecalculation target of the AF evaluation value are selected from thesampling frames set in advance.

In the example of FIG. 13A, out of the sampling frames set in advance, apart of the sampling frames included in the effective region are set asthe evaluation value calculation target region. In addition, in theexample of FIG. 13B, since all of the sampling frames set in advance areincluded in the effective region, all of the sampling frames are set asthe evaluation value calculation target region.

Note that, although all of the sampling frames included in the effectiveregion are set as the evaluation value calculation target region in theexample of FIGS. 13A and 13B, a part of the sampling frames included inthe effective region may also be set as the evaluation value calculationtarget region.

Information regarding such an evaluation value calculation target regionis supplied from the lens controller 136 to the AF sampling unit 135,and the AF evaluation value is calculated on the basis of the pixelsignals of the pixels of the sampling frames set as the evaluation valuecalculation target region. In addition, the lens controller 136 performsthe autofocus processing on the basis of the calculated AF evaluationvalue.

As a result, the CCU 12 can reliably focus on the surgical treatmentportion imaged on the effective region.

Fifth Embodiment: Exemplary Autofocus Processing

An F-number of the scope 52 may be estimated on the basis of the scopediameter, the center position of the effective region, and the positionof the mask region specified as described above, and a depth of focusmay be obtained on the basis of the estimated F-number.

The F-number and the depth of focus of the scope 52 are used by the lenscontroller 136 in order to set parameters for defining details of theautofocus processing, such as an AF speed, focus accuracy, and awobbling amplitude, for example. The lens controller 136 calculates eachof the AF speed, the focus accuracy, and the wobbling amplitude on thebasis of the F-number and the depth of focus of the scope 52 andperforms the autofocus processing on the basis of the calculationresult.

As a result, the lens controller 136 can perform the autofocusprocessing with higher accuracy. Note that a table regarding arelationship between the F-numbers, the depths of focus of the scope,and the scope diameters may be stored in advance, and the F-numbers andthe depths of focus may be obtained by referencing the table.

Operation of CCU

AF Processing

An AF processing of the CCU 12 will be described with reference to theflowchart of FIG. 14.

The process of FIG. 14 starts, for example, when the endoscope 19photographs a surgical treatment portion, and a photographic signal issupplied from the imaging element 111.

In step S1, the camera signal processing unit 131 applies various typesof signal processings such as the white balance processing to thephotographic signal supplied from the imaging element 111.

In step S2, the sampling frame gate 132 outputs the photographic signalof the pixel of each sampling frame.

For example, in a case where the mask region is detected as described inconjunction with FIGS. 7 and 8, the photographic signals of the pixelsof each sampling frame set for the entire photographic image are outputfrom the sampling frame gate 132 and are supplied to the sampling unit133.

In addition, in a case where the mask region is detected as described inconjunction with FIGS. 9, 10, and 11, the photographic signals of thepixels of each sampling frame set in predetermined positions of thephotographic image are output from the sampling frame gate 132 and aresupplied to the AF sampling unit 135.

In step S3, a mask region detection process is performed. Informationregarding the scope diameter, the center position of the effectiveregion, and the position of the mask region specified in the mask regiondetection process is supplied from the mask detection unit 134 to thelens controller 136. Details of the mask region detection process willbe described below with reference to the flowcharts of FIGS. 15 and 16.

In step S4, the lens controller 136 sets the evaluation valuecalculation target region on the basis of the detection result of themask region.

In a case where the evaluation value calculation target region is set asdescribed in conjunction with FIGS. 12A and 12B, the lens controller 136outputs information regarding the evaluation value calculation targetregion to the sampling frame gate 132 and sets the sampling frames inthe evaluation value calculation target region.

In addition, in a case where the evaluation value calculation targetregion is set as described in conjunction with FIGS. 13A and 13B, thelens controller 136 outputs information regarding the evaluation valuecalculation target region to the AF sampling unit 135 and calculates theAF evaluation value for the sampling frame selected as the evaluationvalue calculation target region.

In step S5, the AF sampling unit 135 calculates the AF evaluation valuefor the sampling frame of the evaluation value calculation target regionas a target on the basis of the photographic signal supplied from thesampling frame gate 132.

In step S6, the lens controller 136 performs the autofocus processing onthe basis of the AF evaluation value calculated by the AF sampling unit135. A control signal is output from the lens controller 136 to the lensdriver 113 of the camera head 51 to adjust the position of the focuslens 102. As a result, the AF is implemented.

The aforementioned processes are repeated while the photographic signalis supplied from the imaging element 111.

<Mask Region Detection Process>

Next, the mask region detection process performed in step S3 of FIG. 14will be described with reference to the flowchart of FIG. 15.

The process of FIG. 15 is to detect a mask region using the samplingvalue obtained by the sampling unit 133. The photographic signals of thepixels of each sampling frame set for the entire photographic image aresupplied from the sampling frame gate 132 to the sampling unit 133 asdescribed in conjunction with FIGS. 7 and 8.

In step S11, the sampling unit 133 samples the photographic signalsupplied from the sampling frame gate 132 and outputs sampling values ofeach sampling frame.

In step S12, the mask detection unit 134 detects a maximum samplingvalue from sampling values of sampling frames of each column. Inaddition, the mask detection unit 134 detects a maximum sampling valuefrom sampling values of sampling frames of each row.

In step S13, the mask detection unit 134 obtains differences betweensampling values of every other column in an array of maximum samplingvalues of each column. In addition, the mask detection unit 134 obtainsdifferences between sampling values of every other row in an array ofmaximum sampling values of each row.

In step S14, the mask detection unit 134 sequentially calls thedifferences between sampling values of every other column and detectspositions where a difference equal to or larger than a threshold valueis obtained as left and right edge positions of the mask region. Inaddition, the mask detection unit 134 sequentially calls the differencesbetween sampling values of every other row and detects positions where adifference equal to or larger than a threshold value is obtained asupper and lower edge positions of the mask region.

In step S15, the mask detection unit 134 specifies the scope diameterand the center position of the effective region on the basis of the edgewidth of the effective region.

In step S16, the mask detection unit 134 outputs information regardingthe scope diameter, the center position of the effective region, and theposition of the mask region to the lens controller 136 as a detectionresult of the mask region. Then, the process returns to step S3 of FIG.14, and the subsequent processes are performed.

Next, another mask region detection process performed in step S3 of FIG.14 will be described with reference to the flowchart of FIG. 16.

The process of FIG. 16 is a process of detecting the mask region usingthe AF evaluation value obtained by the AF sampling unit 135.

In step S21, the lens controller 136 outputs information regarding thepositions of the sampling frames to the sampling frame gate 132 and setsthe sampling frames. For example, information for setting the samplingframes in each position of the photographic image as described inconjunction with FIGS. 9, 10, and 11 is supplied from the lenscontroller 136 to the sampling frame gate 132. When the sampling frameis set, the photographic signal corresponding to the pixel of thesampling frame is supplied from the sampling frame gate 132 to the AFsampling unit 135.

In step S22, the AF sampling unit 135 calculates the AF evaluation valueon the basis of the photographic signal supplied from the sampling framegate 132. The calculated AF evaluation value is supplied to the maskdetection unit 134.

In step S23, the mask detection unit 134 determines whether or not achange equal to or larger than a threshold value is generated in the AFevaluation value. If it is determined that a change equal to or largerthan the threshold value is not generated in step S23, the processreturns to step S21, and the aforementioned process is repeated aftersetting the sampling frame by changing the position or width.

Otherwise, if it is determined that a change equal to or larger than thethreshold value is generated in the AF evaluation value in step S23, themask detection unit 134 specifies the position where the change equal toor larger than the threshold value is generated in the AF evaluationvalue as the edge position of the mask region in step S24.

The process after specifying the edge position of the mask region issimilar to that subsequent to step S15 in FIG. 15. Specifically, in stepS25, the mask detection unit 134 specifies the scope diameter and thecenter position of the effective region on the basis of the edge widthof the effective region.

In step S26, the mask detection unit 134 outputs information regardingthe scope diameter, the center position of the effective region, and theposition of the mask region to the lens controller 136 as a detectionresult of the mask region. Then, the process returns to step S3 of FIG.14, and the subsequent process is performed.

Through the aforementioned processes, the CCU 12 can set the calculationtarget region for the AF evaluation value in an appropriate position ofthe effective region with a suitable size depending on the type of thescope 52 installed in the endoscope 19 which is a rigid endoscope.

In addition, the CCU 12 can suitably set parameters such as the AF speedor the wobbling amplitude depending on a specification of the scope 52and thus improve the AF performance. Since the AF speed and the wobblingamplitude are set depending on the specification of the scope 52, it ispossible to avoid degradation of image quality such as a deviation inthe AF operation, a blurry stop phenomenon, and visualization of awobbling operation.

For example, if a large diameter scope is installed, and the AFoperation is performed using an F-number and a depth of focus largerthan suitable values, the AF speed becomes too fast, and a deviationoccurs in the AF operation. In addition, a minute vibration in thewobbling operation is visualized on the output image disadvantageously,for example.

In comparison, if a small diameter scope is installed, and the AFoperation is performed using an F-number and a depth of focus smallerthan suitable values, the AF speed becomes too slow, and the focusing isdelayed disadvantageously. In addition, since the amplitude of thewobbling operation is short, it is difficult to obtain a focusdirection, and a so-called blurry stop in which the AF operation stopsin a blurry screen state occurs disadvantageously.

If the AF operation is performed using a suitable value depending on thespecification of the scope, it is possible to avoid such disadvantagesthat may occur due to an inappropriate specification of the scope.

That is, the endoscope system 1 can provide an operator with anendoscopic image suitable for a surgical operation.

Other Examples Application Example to AE

This embodiment is also applicable to the automatic exposure (AE)processing for performing the AE on the basis of the detection result ofthe mask region. The AE is performed by automatically adjustingparameters such as a shutter speed and an ISO sensitivity without anoperator's manipulation.

FIG. 17 is a block diagram illustrating an exemplary configuration ofthe CCU 12 for performing the AE.

In FIG. 17, like reference numerals denote like elements as in FIG. 5.Some components will not be repeatedly described. The configuration ofthe CCU 12 of FIG. 17 is different from that of FIG. 5 in that anexposure detection unit 141 is provided.

Out of the photographic signals supplied from the camera signalprocessing unit 131, the sampling frame gate 132 outputs a photographicsignal of the pixel of the sampling frame used for exposure evaluationto the exposure detection unit 141.

The exposure detection unit 141 calculates the AE evaluation value as anevaluation value for exposure evaluation on the basis of thephotographic signal of the pixel of the sampling frame and outputs theAE evaluation value to the lens controller 136.

The lens controller 136 outputs a control signal for adjusting theshutter speed or the ISO sensitivity to the imaging element driver 112on the basis of the AE evaluation value calculated by the exposuredetection unit 141 to perform the automatic exposure processing.

FIG. 18 is a diagram illustrating an exemplary setting of the evaluationvalue calculation target region for the AE evaluation value.

The setting of the evaluation value calculation target region of FIG. 18corresponds to the setting of the evaluation value calculation targetregion described in conjunction with FIGS. 12A and 12B.

As illustrated in the right half of FIG. 18, the lens controller 136sets the evaluation value calculation target region serving as acalculation target for the AE evaluation value inside the effectiveregion so as not to overlap with the mask region. For example, theevaluation value calculation target region is set by magnifying orreducing the size of the default area and shifting the default centerposition on the basis of the detection result of the mask region.

The oblong rectangular area A11 illustrated in the right half of FIG. 18is an evaluation value calculation target region set inside theeffective region having a diameter shorter than the vertical length ofthe photographic image. In the example of FIG. 18, for example, theevaluation value calculation target region is set by reducing thedefault size illustrated in the left half of FIG. 18.

Information regarding such an evaluation value calculation target regionis supplied from the lens controller 136 to the sampling frame gate 132,and the sampling frames serving as a calculation target of the AEevaluation value are set inside the evaluation value calculation targetregion.

The exposure detection unit 141 calculates the AE evaluation value onthe basis of the photographic signals of the pixels of the samplingframes set inside the evaluation value calculation target region. Inaddition, the lens controller 136 performs the automatic exposureprocessing on the basis of the calculated AE evaluation value.

As a result, the CCU 12 can appropriately adjust the exposure of thesurgical treatment portion imaged on the effective region.

For example, in a case where the sampling frames for exposure evaluationare set in the mask region, a control for increasing exposure isperformed in the AE, and as a result, exposure may become excessive insome cases. By evaluating the exposure only using the sampling framesinside the effective region, the CCU 12 can adjust a brightness of thesurgical treatment portion to suitable exposure.

FIG. 19 is a diagram illustrating another exemplary setting of theevaluation value calculation target region for the AE evaluation value.

The setting of the evaluation value calculation target region of FIG. 19corresponds to the setting of the evaluation value calculation targetregion described in conjunction with FIGS. 13A and 13B.

Each segment of the mesh pattern overlapping on the photographic imageof the left half of FIG. 19 represents a sampling frame. In the exampleof FIG. 19, a plurality of sampling frames (ten sampling frames in thevertical direction and ten frames in the horizontal direction) are setside by side in a matrix shape around the center of the photographicimage. In this example, the positions of the sampling frames are fixedregardless of the range of the effective region. The photographicsignals of the pixels of each sampling frame are supplied from thesampling frame gate 132 to the exposure detection unit 141.

Out of the sampling frames fixedly set in this manner, the lenscontroller 136 selects the sampling frames entirely included in theeffective region without overlapping with the mask region as theevaluation value calculation target region serving as a calculationtarget for the AE evaluation value. The sampling frames colored in theright half of FIG. 19 are sampling frames selected as the evaluationvalue calculation target region. Meanwhile, the sampling framesoverlapping with the mask region are treated as invalid sampling frames(having a weight of zero).

The lens controller 136 outputs information regarding the sampling frameselected as the evaluation value calculation target region to theexposure detection unit 141 to calculate the AE evaluation value for thesampling frame selected as the evaluation value calculation targetregion.

Information regarding such an evaluation value calculation target regionis supplied from the lens controller 136 to the exposure detection unit141, and the AE evaluation value is calculated on the basis of the pixelsignals of the pixels of the sampling frames set as the evaluation valuecalculation target region. In addition, the lens controller 136 performsthe automatic exposure processing on the basis of the calculated AEevaluation value.

As a result, the CCU 12 can suitably adjust exposure of the surgicaltreatment portion imaged on the effective region.

The automatic exposure processing may be performed in combination withthe autofocus processing described above. Specifically, the controlprocess of the lens controller 136 is a process including at least oneof the automatic exposure processing or the autofocus processing.

Application Example to AWB

The white balance processing for performing an auto white balance (AWB)of the endoscope system 1 may be executed by the camera signalprocessing unit 131 on the basis of the detection result of the maskregion.

In this case, the detection result of the mask region is supplied fromthe mask detection unit 134 to the camera signal processing unit 131,and the white balance processing is performed on the basis of thephotographic signals of the pixels of the effective region.

The white balance processing is a process of correcting colors of thephotographic image to image the surgical treatment portion with suitablecolor tones. For example, a photographic environment is estimated on thebasis of the photographic signals (color signals) of the pixels of theeffective region, and the colors generated from the photographic signalsare corrected.

In addition, the white balance processing may be performed such that acolor temperature of the light source is estimated from the photographicsignals of the pixels of the effective region, and the colors arecorrected depending on the color temperature of the light source storedin advance.

In addition, if it is determined that red light is weak inside theeffective region, the white balance processing may be performed toemphasize the red light in order to highlight a blood vessel.

If the white balance processing is performed using the photographicsignals of the pixels of the effective region in this manner, it ispossible to obtain the photographic image on which the surgicaltreatment portion is imaged with appropriate color tones.

Other Examples

While a case where the AF, AE, and AWB based on the detection result ofthe mask region are applied to the endoscope system 1 has beendescribed, the present technology may also be applicable to a case wherethe AF, AE, and AWB are performed in the microscope system.

The mask region is detected on the basis of the sampling value obtainedby the sampling unit 133 or the AF evaluation value obtained by the AFsampling unit 135 in the aforementioned description. Alternatively, themask region may be detected on the basis of both the sampling value andthe AF evaluation value.

A series of processes described above may be executed using eitherhardware or software. In a case where a series of processes are executedusing software, a program included in this software is installed from aprogram recording medium to a computer integrated to dedicated hardware,a general-purpose personal computer, and the like.

FIG. 20 is a block diagram illustrating an exemplary hardwareconfiguration of the computer for executing a series of processesdescribed above using a program.

A central processing unit (CPU) 1001, a read only memory (ROM) 1002, anda random access memory (RAM) 1003 are connected to each other via a bus1004.

An input/output interface 1005 is further connected to the bus 1004. Aninput unit 1006 such as a keyboard or a mouse and an output unit 1007such as a display or a loudspeaker are connected to the input/outputinterface 1005. In addition, a storage unit 1008 such as a hard disk ora non-volatile memory, a communication unit 1009 such as a networkinterface, and a drive 1010 for driving a removable medium 1011 areconnected to the input/output interface 1005.

In the computer configured in this manner, the CPU 1001 performs aseries of processes described above by loading and executing, forexample, the program stored in the storage unit 1008 on the RAM 1003 viathe input/output interface 1005 and the bus 1004.

The program executed by the CPU 1001 is recorded, for example, on theremovable medium 1011 or is provided via a wired or wirelesstransmission medium such as a local area network, the Internet, anddigital broadcasting, and is installed in the storage unit 1008.

Note that the program executed by the computer may be a programprocessed in a time-series manner depending on the sequence describedherein or a program processed in parallel or at necessary timings suchas in response to a call.

Note that, herein, a system refers to a set of a plurality ofconstituent elements (devices, modules (components), or the like)regardless of whether or not all of them are integrated into the samecasing. Therefore, the system encompasses a plurality of devices housedin separate casings and connected via a network, a single device havinga plurality of modules housed in a single casing, and the like.

Note that the advantageous effects described herein are just forexemplary purposes and are not construed in a limitative sense. Anotheradvantageous effect may also be included.

Embodiments of the present technology are not limited to those describedabove, and various changes or modifications may be possible withoutdeparting from the scope and spirit of the present technology.

For example, the present technology may be applicable to a cloudcomputing environment in which a single function is processed in adistributed and cooperated manner across a plurality of devicesconnected via a network.

In addition, each step of the flowchart described above may be executedusing a single device or using a plurality of devices in a distributedmanner.

In addition, in a case where a plurality of processes are included in asingle step, they may be executed using a single device or using aplurality of devices in a distributed manner.

<Combination of Configurations>

The present technology may also include the following configurations.

(1) According to one aspect, a system includes an endoscope including ascope and an image sensor, the image sensor being configured to capturemedical image data that includes effective image portion data and amechanical vignetting portion data, the mechanical vignetting portiondata of the medical image data being generated due to mechanicalvignetting caused by a difference in the image sensor which generatesthe medical image data and the scope; and circuitry configured todetermine evaluation information from image data which is from theeffective image portion data, and execute a control process to at leastpartially control at least one of an autofocus processing, and an autowhite balance processing on the endoscope on the basis of the evaluationinformation.

(2) The system according to (1), wherein the determining by thecircuitry of the evaluation information determines the evaluationinformation from only the medical image data which is from the effectiveimage portion data.

(3) The system according to (2), wherein the difference is due to adifferent shape of the image sensor and a light passed through thescope.

(4) The system according to (3), wherein the image sensor is rectangularand the effective image portion data is of circular image.

(5) The system according to (2), wherein the difference is due adifferent size of the image sensor and the scope.

(6) The system according to (5), wherein an image from the scope whichis used to generate the effective image portion data exceeds a size ofan effective area of the image sensor in a first dimension of the imagesensor, is smaller than a size of the effective area of the image sensorin a second dimension of the image sensor which is perpendicular to thefirst dimension, and the first dimension and the second dimension areboth in an imaging sensing plane of the image sensor.

(7) The system according to (2), further including a light whichilluminates a target which corresponds to the effective image portiondata.

(8) The system according to (2), further including a lens system,wherein the executing of the control process by the circuitry generatesinformation to focus the lens system on the basis of the evaluationinformation.

(9) The system according to (2), wherein the executing of the controlprocess by the circuitry generates white balance information to set awhite balance of the medical image.

(10) The system according to (1), wherein the circuitry is furtherconfigured to sample luminance signals from the image sensor, anddetermine where the effective image portion data exists and theeffective image portion data using the luminance signals.

(11) The system according to (10), wherein the determining of where theeffective image portion data exists detects an edge of the effectiveimage portion data on the basis of the luminance signals which have beensampled and are in a matrix shape, and specifies a diameter of the scopeand a center position of the effective image portion data.

(12) The system according to (1), wherein the circuitry is furtherconfigured to perform sampling in sampling frames set on the medicalimage data to obtain the evaluation information which is used for theautofocus processing, and determine where the effective image portiondata exists and the effective image portion data using the evaluationinformation which is used for the autofocus processing, wherein theexecuting of the control process at least partially controls theautofocus processing based on the evaluation information.

(13) The system according to (12), wherein the performing of thesampling in sampling frames changes at least one of a position and asize of the sampling frames.

(14) The system according to (12), wherein the determining of where theeffective image portion data exists specifies a diameter of the scopeand a center position of the effective image portion data.

(15) The system according to (1), wherein the circuitry is furtherconfigured to set an evaluation value calculation target region used asthe effective image portion using a diameter of the scope and a centerposition of the effective image portion data.

(16) The system according to (1), wherein the executing of the controlprocess executes the control process on the basis of the evaluationinformation which is determined based on the effective image portiondata indicated by a diameter of the scope and a center position of theeffective image portion data out of evaluation value calculation targetregions set in a matrix shape on the medical image data as a calculationtarget for the evaluation information.

(17) The system according to (1), wherein the determining of theevaluation information determines information used for the autofocusprocessing on the basis of at least one of an F-number or a depth offocus of the scope estimated using the image data from the effectiveimage portion data.

(18) The system according to (1), wherein the executing of the controlprocess references a table containing at least one of an F-number or adepth of focus of the scope stored in advance on the basis of adetermining result of the determining of the evaluation information andsets a parameter for defining the autofocus processing on the basis of aresult of the referencing.

(19) A system for processing medical images, including circuitryconfigured to obtain medical image data by an endoscope including animager head and a scope attached to the imager head, determineevaluation information using effective image area data of the medicalimage data without using mechanical vignetting area data of the medicalimage data, the effective area of the medical image data being generateddue to mechanical vignetting caused by the scope, and execute a controlprocess including at least one of an autofocus process and an auto whitebalance process on the basis of the evaluation information.

(20) The system according to (19) further including the imager headwhich includes an image sensor; and the scope which is a medicalinstrument, wherein the mechanical vignetting area data is due to adifferent shape of the image sensor and the scope.

(21) The system according to (20) wherein the image sensor isrectangular and the medical image data produced by the scope is of acircular image.

(22) The system according to (19), further including the imager headwhich includes an image sensor; and the scope which is a medicalinstrument, wherein the mechanical vignetting area data is due to adifferent size of the image sensor and the scope.

(23) The system according to (22), wherein a medical image produced bythe endoscope exceeds a size of the image sensor in a first dimension ofthe image sensor, is smaller than a size of the image sensor in a seconddimension of the image sensor which is perpendicular to the firstdimension, and the first dimension and the second dimension are both inan imaging sensing plane of the image sensor.

(24) The system according to (19), further including a light whichilluminates a target which corresponds to the medical image data.

(25) The system according to (19) further including a lens systemattached to the scope, wherein the circuitry configured to execute thecontrol process generates information to focus the lens system on thebasis of the evaluation information.

(26) The system according to (19), wherein the circuitry configured toexecute the control process generates white balance information to set awhite balance of the medical image.

(27) A method of processing medical image information includingdetermining evaluation information using effective image portion data ofmedical image data, the medical image data including the effective imageportion data and mechanical vignetting portion data, the mechanicalvignetting portion data of the medical image data being generated due tomechanical vignetting caused by a difference in an image sensor whichgenerates the medical image data and a medical instrument, executing acontrol process including at least one of an autofocus process, or anauto white balance process on the basis of the evaluation information.

(28) The method according to (27), wherein the determining of theevaluation information determines the evaluation information from onlythe effective image portion data of the medical image data without usingthe mechanical vignetting portion data.

(29) The method according to (28), further including generating themedical image data using the image sensor and the medical instrument,wherein the difference is due a different shape of the image sensor andthe medical instrument.

(30) The method according to (29), wherein the image sensor isrectangular, and an image corresponding to the effective image portiondata produced by the medical instrument is circular.

(31) The method according to (28), further including generating themedical image data using the image sensor and the medical instrument,wherein the difference is due a different size of the image sensor andthe medical instrument.

(32) The method according to (31), wherein

an image from the medical instrument which is used to generate theeffective image portion data exceeds a size of the image sensor in afirst dimension of the image sensor, is smaller than a size of the imageproduced by the medical instrument in a second dimension of the imagesensor which is perpendicular to the first dimension, and the firstdimension and the second dimension are both in an imaging sensing planeof the image sensor.

(33) The method according to (28), wherein the medical instrumentincludes a scope.

(34) The method according to (28), further including generating themedical image data using the image sensor and the medical instrumentwhich includes a scope.

(35) The method according to (34), further including illuminating, usinga light, a target which corresponds to the medical image.

(36) The method according to (27), wherein the executing of the controlprocess executes a focusing of a lens system using the evaluationinformation.

(37) The method according to (27), wherein the executing of the controlprocess executes the auto white balance process using the evaluationinformation.

(38) The method according to claim (27), further including samplingluminance signals from the image sensor, and determining where theeffective image portion data exists and the effective image portion datausing the luminance signals.

(39) The method according to (38), wherein the determining of where theeffective image portion data exists detects an edge of the effectiveimage portion data on the basis of the luminance signals which have beensampled and are in a matrix shape, and specifies a diameter of the scopeand a center position of the effective image portion data.

(40) The method according to (27), further including

performing sampling in sampling frames set on the medical image data toobtain the evaluation information which is used for the autofocusprocessing, and determining where the effective image portion dataexists and the effective image portion data using the evaluationinformation which is used for the autofocus process, wherein theexecuting of the control process at least partially controls theautofocus process based on the evaluation information.

(41) The method according to (40), wherein:

the performing of the sampling in sampling frames changes at least oneof a position and a size of the sampling frames.

(42) The method according to (40), wherein:

the determining of where the effective image portion data existsspecifies a diameter of the scope and a center position of the effectiveimage portion data.

(43) The method according to (27), further comprising:

setting an evaluation value calculation target region used as theeffective image portion using a diameter of the scope and a centerposition of the effective image portion data.

(44) The method according to (27), wherein the executing of the controlprocess executes the control process on the basis of the evaluationinformation which is determined based on the effective image portiondata indicated by a diameter of the scope and a center position of theeffective image portion data out of evaluation value calculation targetregions set in a matrix shape on the medical image data as a calculationtarget for the evaluation information.

(45) The method according to (27), wherein the determining of theevaluation information determines information used for the autofocusprocessing on the basis of at least one of an F-number or a depth offocus of the scope estimated using the image data from the effectiveimage portion data.

(46) The method according to (27), wherein the executing of the controlprocess references a table containing at least one of an F-number or adepth of focus of the scope stored in advance on the basis of adetermining result of the determining of the evaluation information andsets a parameter for defining the autofocus processing on the basis of aresult of the referencing.

<Combination of Configurations>

The present technology may also include the following configurations.

(1)

An endoscope system including:

a light source device configured to irradiate light onto a surgicalfield area;

an image sensing device configured to photograph the surgical field areausing a detachably installed scope; and

an information processing device connected to the image sensing deviceand the light source device and provided with

a detection unit configured to detect an effective region of the scopefrom a photographic image photographed by the image sensing device, and

a control unit configured to execute a control process including atleast one of an autofocus processing, an automatic exposure processing,or an auto white balance processing on the basis of an evaluation valueof the effective region.

(2)

The endoscope system according to (1), in which the effective region hasno vignetting caused by the scope.

(3)

The endoscope system according to (1) or (2), further including a firstsampling unit configured to sample luminance signals of each region setin the photographic image, in which the detection unit detects theeffective region on the basis of sampling values of each region of thephotographic image.

(4)

The endoscope system according to (3), in which the detection unitdetects an edge of the effective region on the basis of the samplingvalue of each region set in a matrix shape on the photographic image andspecifies a diameter of the scope and a center position of the effectiveregion.

(5)

The endoscope system according to (1), further including a secondsampling unit configured to perform sampling in sampling frames set onthe photographic image and calculate an evaluation value for theautofocus processing,

in which the detection unit detects the effective region on the basis ofthe evaluation value for the autofocus processing, and

the control unit executes the autofocus processing as the controlprocess on the basis of the evaluation value for the autofocusprocessing.

(6)

The endoscope system according to (5), in which the second sampling unitcalculates the evaluation value for the autofocus processing used indetection of the effective region by changing at least one of a positionor a size of the sampling frame.

(7)

The endoscope system according to (6), in which the detection unitdetects an edge of the effective region on the basis of the evaluationvalue for the autofocus processing and specifies the diameter of thescope and the center position of the effective region.

(8)

The endoscope system according to any of (1) to (7), in which thecontrol unit sets an evaluation value calculation target region servingas a calculation target for the evaluation value in the effective regionindicated by the diameter of the scope and the center position of theeffective region.

(9)

The endoscope system according to any of (1) to (8), in which thecontrol unit executes the control process on the basis of the evaluationvalue calculated for the evaluation value calculation target region ofthe effective region indicated by the diameter of the scope and thecenter position of the effective region out of the evaluation valuecalculation target regions set in a matrix shape on the photographicimage as a calculation target for the evaluation value.

(10)

The endoscope system according to any of (1) to (9), in which thecontrol unit sets a parameter for defining the autofocus processing onthe basis of at least one of an F-number or a depth of focus of thescope estimated on the basis of a detection result of the detectionunit.

(11)

The endoscope system according to any of (1) to (9), in which thecontrol unit references a table containing at least one of an F-numberor a depth of focus of the scope stored in advance on the basis of adetection result of the detection unit and sets a parameter for definingthe autofocus processing on the basis of a result of the referencing.

(12)

A method of controlling an endoscope system, the method including:

irradiating light onto a surgical field area using a light sourcedevice;

photographing the surgical field area through a detachably installedscope using an image sensing device; and

using an information processing device connected to the image sensingdevice and the light source device to detect an effective region of thescope from a photographic image photographed by the image sensing deviceand execute a control process including at least one of an autofocusprocessing, an automatic exposure processing, or an auto white balanceprocessing on the basis of an evaluation value of the effective region.

(13)

An information processing device of an endoscope system, the informationprocessing device being connected to a light source device configured toirradiate light onto a surgical field area and an image sensing deviceconfigured to photograph the surgical field area through a detachablyinstalled scope, the information processing device including:

a detection unit configured to detect an effective region of the scopefrom a photographic image photographed by the image sensing device; and

a control unit configured to execute a control process including atleast one of an autofocus processing, an automatic exposure processing,or an auto white balance processing on the basis of an evaluation valueof the effective region.

(14)

A program for causing a computer serving as an information processingdevice of an endoscope system to execute a processing, the informationprocessing device being connected to a light source device configured toirradiate light onto a surgical field area and an image sensing deviceconfigured to photograph the surgical field area through a detachablyinstalled scope,

the processing including:

detecting an effective region of the scope from a photographic imagephotographed by the image sensing device; and

executing a control process including at least one of an autofocusprocessing, an automatic exposure processing, or an auto white balanceprocessing on the basis of an evaluation value of the effective region.

REFERENCE SIGNS LIST

-   -   1 Endoscope system    -   12 CCU    -   19 Endoscope    -   51 Camera head    -   52 Scope    -   111 Imaging element    -   112 Imaging element driver    -   113 Lens driver    -   114 Zoom lens driving unit    -   115 Focus lens driving unit    -   131 Camera signal processing unit    -   132 Sampling frame gate    -   133 Sampling unit    -   134 Mask detection unit    -   135 AF sampling unit    -   136 Lens controller    -   141 Exposure detection unit

The invention claimed is:
 1. A system, comprising: an endoscopeincluding a scope and an image sensor, the image sensor being configuredto capture medical image data that includes effective image portion dataand a mechanical vignetting portion data, the mechanical vignettingportion data of the medical image data being generated due to mechanicalvignetting caused by a difference in the image sensor which generatesthe medical image data and the scope; and circuitry configured todetermine evaluation information from image data which is only from theeffective image portion data of the medical image data without using themechanical vignetting portion data, and execute a control process to atleast partially control at least one of an autofocus processing, and anauto white balance processing on the endoscope on the basis of theevaluation information.
 2. The system according to claim 1, wherein: thedifference is due to a different shape of the image sensor and a lightpassed through the scope.
 3. The system according to claim 2, wherein:the image sensor is rectangular and the effective image portion data isof circular image.
 4. The system according to claim 1, wherein: thedifference is due a different size of the image sensor and the scope. 5.The system according to claim 4, wherein: an image from the scope whichis used to generate the effective image portion data exceeds a size ofan effective area of the image sensor in a first dimension of the imagesensor, is smaller than a size of the effective area of the image sensorin a second dimension of the image sensor which is perpendicular to thefirst dimension, and the first dimension and the second dimension areboth in an imaging sensing plane of the image sensor.
 6. The systemaccording to claim 1, further comprising: a light which illuminates atarget which corresponds to the effective image portion data.
 7. Thesystem according to claim 1, further comprising: a lens system, whereinthe executing of the control process by the circuitry generatesinformation to focus the lens system on the basis of the evaluationinformation.
 8. The system according to claim 1, wherein: the executingof the control process by the circuitry generates white balanceinformation to set a white balance of the medical image.
 9. The systemaccording to claim 1, wherein the circuitry is further configured to:sample luminance signals from the image sensor, and determine where theeffective image portion data exists and the effective image portion datausing the luminance signals.
 10. The system according to claim 9,wherein the determining of where the effective image portion data existsdetects an edge of the effective image portion data on the basis of theluminance signals which have been sampled and are in a matrix shape, andspecifies a diameter of the scope and a center position of the effectiveimage portion data.
 11. The system according to claim 1, wherein thecircuitry is further configured to: perform sampling in sampling framesset on the medical image data to obtain the evaluation information whichis used for the autofocus processing, and determine where the effectiveimage portion data exists and the effective image portion data using theevaluation information which is used for the autofocus processing,wherein the executing of the control process at least partially controlsthe autofocus processing based on the evaluation information.
 12. Thesystem according to claim 11, wherein: the performing of the sampling insampling frames changes at least one of a position and a size of thesampling frames.
 13. The system according to claim 11, wherein: thedetermining of where the effective image portion data exists specifies adiameter of the scope and a center position of the effective imageportion data.
 14. The system according to claim 1, wherein the circuitryis further configured to: set an evaluation value calculation targetregion used as the effective image portion using a diameter of the scopeand a center position of the effective image portion data.
 15. Thesystem according to claim 1, wherein: the executing of the controlprocess executes the control process on the basis of the evaluationinformation which is determined based on the effective image portiondata indicated by a diameter of the scope and a center position of theeffective image portion data out of evaluation value calculation targetregions set in a matrix shape on the medical image data as a calculationtarget for the evaluation information.
 16. The system according to claim1, wherein: the determining of the evaluation information determinesinformation used for the autofocus processing on the basis of at leastone of an F-number or a depth of focus of the scope estimated using theimage data from the effective image portion data.
 17. The systemaccording to claim 1, wherein: the executing of the control processreferences a table containing at least one of an F-number or a depth offocus of the scope stored in advance on the basis of a determiningresult of the determining of the evaluation information and sets aparameter for defining the autofocus processing on the basis of a resultof the referencing.
 18. A system for processing medical images,comprising: circuitry configured to obtain medical image data by anendoscope including an imager head and a scope attached to the imagerhead, determine evaluation information using only effective image areadata of the medical image data without using mechanical vignetting areadata of the medical image data, the mechanical vignetting area data ofthe medical image data being generated due to mechanical vignettingcaused by the scope, and execute a control process including at leastone of an autofocus process and an auto white balance process on thebasis of the evaluation information.
 19. The system according to claim18, further comprising: the imager head which includes an image sensor;and the scope which is a medical instrument, wherein the mechanicalvignetting area data is due to a different shape of the image sensor andthe scope.
 20. The system according to claim 19, wherein: the imagesensor is rectangular and the medical image data produced by the scopeis of a circular image.
 21. The system according to claim 18, furthercomprising: the imager head which includes an image sensor; and thescope which is a medical instrument, wherein the mechanical vignettingarea data is due to a different size of the image sensor and the scope.22. The system according to claim 21, wherein: a medical image producedby the endoscope exceeds a size of the image sensor in a first dimensionof the image sensor, is smaller than a size of the image sensor in asecond dimension of the image sensor which is perpendicular to the firstdimension, and the first dimension and the second dimension are both inan imaging sensing plane of the image sensor.
 23. The system accordingto claim 18, further comprising: a lens system attached to the scope,wherein the circuitry configured to execute the control processgenerates information to focus the lens system on the basis of theevaluation information.
 24. The system according to claim 18, wherein:the circuitry configured to execute the control process generates whitebalance information to set a white balance of the medical image.
 25. Amethod of processing medical image information, comprising: determiningevaluation information using only effective image portion data ofmedical image data, the medical image data including the effective imageportion data and mechanical vignetting portion data, the mechanicalvignetting portion data of the medical image data being generated due tomechanical vignetting caused by a difference in an image sensor whichgenerates the medical image data and a medical instrument; and executinga control process including at least one of an autofocus process, or anauto white balance process on the basis of the evaluation information.26. The method according to claim 25, further comprising: generating themedical image data using the image sensor and the medical instrument,wherein the difference is due a different shape of the image sensor andthe medical instrument.
 27. The method according to claim 25, furthercomprising: generating the medical image data using the image sensor andthe medical instrument, wherein the difference is due a different sizeof the image sensor and the medical instrument.
 28. The method accordingto claim 27, wherein: an image from the medical instrument which is usedto generate the effective image portion data exceeds a size of the imagesensor in a first dimension of the image sensor, is smaller than a sizeof the image produced by the medical instrument in a second dimension ofthe image sensor which is perpendicular to the first dimension, and thefirst dimension and the second dimension are both in an imaging sensingplane of the image sensor.